■ DOCUMENT RESUME 
ED 381 88A EA 026 671 



AUTHOR 
TITLE 

INSTITUTION 



SPONS AGENCY 

PUB DATE 
CONTRACT 
NOTE 
PUB TYPE 



Marks, Helen M. 

Student Engagement in the Classrooms of Restructuring 
Schools . 

Center on Organization and Restructuring of Schools, 
Madison, Wl.; Wisconsin Center for Education 
Research, Madison. 

Office of Educational Research and Improvement (ED), 
Washington, DC. 
Jan 95 

R117Q00005-94 
59p. 

Speeches/Conference Papers (150) -- Reports - 
Research. 'Technical (143) 



EDRS PRICE MF01/PC03 Plus Postage. 

DESCRIPTORS Academic Achievement; '-Educat i on.i 1 Environment; 

Educational Innova !" '. on ; Elementary Secondary 
Education; "Instructional Innovation; Mathematics 
Achievement; "School Restructuring; Social Sciences; 
"'■'Student Motivation; "Student Participation 

IDENTIFIERS "Authentic Assessment 



ABSTRACT 

This paper presents findings of a study that 
investigated factors that contribute to the engagement of students. 
Based on the theoretical model of Newmann, Wehlage, and Lamborn 
(1992j, the study examined the effect of instructional approaches of 
teachers, contextual features of classrooms, and attributes of 
student experience on students' engagement in academic work. The 
sample, obtained through a nationwide selection process, included 24 
public schools (8 elementary, 8 middle, and 8 high schools) deeply- 
involved in restructuring. Data were collected from a survey of 5th-, 
8th-, and lOth-grade students and their teachers from six core 
classrooms (3 mathematics pnd 3 social sciences) in each of the 24 
schools. Over 3,660 students in 143 of the 149 core classrooms 
completed surveys. Two-way analysis of variance (ANOVA) and 
hierarchical linear modeling (HLM) were used to analyze the data. 
Finding confirmed the importance of authentic work and structures of 
support tor students. Student self-concept had a moderate effect on 
engagement, and girls were more engaged than v;ere boys. Students in 
mathematics classes had higher levels of engagement than those in 
social studies classes. Surprisingly, the use of traditional 
instructional methods exerted a positive influence on student 
engagement, while progressive methods did rot. The data indicate that 
progressive methods must be combined with cliallenging content and a 
clear focus of inquiry. Specific restructuring content, such as 
authentic academic work and structures of support for learning, 
proved important in raising student engagement even where generalized 
restructuring v;as taking place. Two figures and six tables are 
included. Appendices contain statistical data and information on the 
construction of vrjr'ables and mo t liodo 1 ogy . ( LM 1 ) 
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Student Engagement in the Classrooms of Restructuring Schools 



In the mid-1980s a series of field studies (e.g., Cusick, 1983; Goodlad, 1984; Oakes, 1985; 
Powell, Farrar, and Cohen, 1986; Sizer, 1985; Sedlak, Wheeler, Pullin, & Cusick, 1986) presented a 
troubling picture of the internal organization of high schools as routinized bureaucracies where, for 
the most part, dispirited teachers and disengaged students "put in their time," while negotiating a 
sprawling and fiagmented curriculum. Variously typified as ennui, malaise, alienation, anomie - the 
student disengagement these researchers portrayed has come to be recognized as a deep and pervasive 
problem, particularly in U.S. high schools (Newmann, 1992). Chronic disengagement reportedly 
afflicts at least two-thirds of high school students (Sedlak et al., 1986), an estimate that excludes large 
proportions of repeated absentees and dropouts.' Lack of engagement with academic work during 
high school adversely affects smdent achievement and initiates a downward spiral that may lead to 
dysfunctional school behavior and, ultimately, culminate hi some students leaving school entirely 
(Finn, 1989; Wehlage, Rutter, Smith, Lesko & Fernandez, 1989). Although no comparable estimate 
of disengagement exists for students in elementary schools, critics have indicted these schools for 
meaningless instructional activities that disillusion students about the usefulness of school, while 
failing to equip them with the skills they need to succeed in secondary school (Finn, 1993; Goodlad; 
Sedlak et al.). 

If anomie is characteristic of U.S. education for the vast majority of students, under what 
conditions are they likely to become engaged? While classrooms where student engagement was high 
constituted a very small segment of the classrooms visited by the authors of the mid-1980s field 
studies, s ich classrooms did exist. When classroom involvement occurred, observers ascribed it to 
the high-track status of the class, students' self-selection into a program of special interest to them, 
or, occasionally, teachers' instructional styles, including the expectations they put forth for students 
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(Oakes, 1985; Powell et al., 1986). Despite the prevailing academic lassitude, the observers noted 
that within classrooms students sometimes varied in their degree of disengagement. While the 
majority of students seemed passive and bored, and some evidenced disdain reflecting a more 
profound alienation, in the same classroom settings, other students appeared somewhat interested or 
even involved — variation Oakes (1985) attributed to an interplay of personal characteristics, school 
experience, and treatment in the classroom. 

Approaching the problem of student disengagement theoretically, Newmann (1981, 1989) 
drew upon sociological and psychological literature on alienation to develop a conceptual model for 
engagement, Newmann located the sources of student disengagement in alienating characteristics of 
bureaucratically organized schools — namely, meaningless, low-level school work and impersonal 
relationships with teachers and other students. For students to become academically engaged, 
therefore, would require a reversal of the alienating experiences: that is, a sense of membership in 
school to replace impp-rsonality and isolation, and authentic academic work to replace low-level school 
work (Newmann, 1989; Newmann, Wehlage & Lambom, 1992). Authentic academic work involves 
students intellectually in a process of disciplined inquiry to solve meaningful problems, that is, 
problems with relevance in the world beyond the classroom and of interest to them personally. 

Because of their logical relationship to achievement, student engagement with schooling and 
with academic work emerged as valued student outcomes for school restructuring (Elmore, 1990; 
Murphy, 1991; Newmann, 1991a). Recent research on restructuring in middle schools and in high 
schools (based on data from NELS:88 and the first foUowup) has documented a positive relationship 
between school restructuring and student engagement (Lee & Smith, 1993, 1994). Because 
restructuring schools are moving away from bureaucratic organization to more organic organizational 
forms, the organizational shift may be contributing to student engagement and achievement gains (Lee 
& Smith). 



Taking a close look at a nationally selected sample of elementary, middle, and high schools 
deeply immersed in restructuring (Berends & King, 1994), the present study investigates factors that 
contribute to the engagement of students in the social studies and mathematics classrooms of these 
schools. Since restructuring schools are likely to be making efforts to improve instruction and 
students' experience of school, the investigation examines instructional approaches of teachers, 
contextual features of classrooms, and attributes of student experience that may differentially influence 
students' engagement in academic work. Guiding the investigation is a modified version of the model 
for student engagement articulated by Newmann (1989) and Newmann, Wehlage & Lambom (1992). 

Background 

Institutional Sources of Student Disengagement 

Many students lack extrinsic motivation for serious engagement with schooling. Despite 
their academic disengagement throughout the school years, U.S. public education ritually promotes 
students from grade to grade and, ultimately, awards a diploma to those who persevere through high 
school. Representing attendance rather than achievement, such symbolic recognition reveals little 
about what students know or can do. Because the diploma has become devalued, high school has lost 
meaning educationally (Sedlak et al., 1986; Sizer, 1985). Employers rarely ask high schools to 
pfo\'ide transcripts or to recommend students for possible employment (Bishop, 1989). Unless 
students are aspiring to highly selective colleges or universities with rigorous admissions criteria, 
incentives for students to achieve in high school are weak (Cohen, 1990). 

Instruction. The failure of U.S. public education to engage students may also be a 
consequence of institutional inattention to the primary function of schools, instruction. An attenuated 
focus on the central mission is characteristic of bureaucratic organizations (Meyer and Rowan, 1978; 



Zucker, 1987). Instruction is weakly normed compared to the institutional culture for bureaucracy, 
which attempts to ensure efficiency, order, and predictability in school operation. While schools 
attend carefiilly to "certifying" and "classifying" (ritual institutional and bureaucratic functions), these 
processes are disembedded from instruction and its expected concomitants - student achievement and 
engagement. Despite being the central activity of schools and integral to their social function, 
iastruction is generally assumed to be adequate, but is only loosely coordinated and rarely inspected 
(Meyer & Rowan). 

Through their portrayal of intellectual vapidity in classrooms across the nation, the field 
studies of the 1980s exposed weaknesses and deficiencies in the "technical core" of schools (e.g., 
instruction, teaching and learning), which is normally shrouded in a "logic of confidence" (Meyer & 
Rowan, 1978). In most of the classrooms visited by the field researchers, instruction, typically 
fc;io;ving the transmission model, induced passivity among students. According to the transmission 
model, the teacher imparts information largely through lecture and the students take notes, recite, and 
fii'l out worksheets. Emphasizing low-level intellectual skills of repetition and recall, the transmission 
model breeds boredom and intellectual isolation (Coleman, 1961; Newmann, 1989). As typically 
enacted, therefore, school work is alienating (Carnoy and Levin, 1985). Alienating work, 
characterized by Marx (1982), is external to the worker's nature; forced rather than voluntary; and 
instrumental, rather than intrinsically motivated. Authentic work, in contrast, "entails extrinsic 
rewards, meets intrinsic interests, offers students a sense of ownership, is connected to the 'real 
world' (i.e., the vorld beyond school)" (Newmann, Wehlage & Lamborn, 1992, p. 23). 

The bargain . Despite finding schoolwork typically irrelevant and unrewarding, students 
submit themselves to it, enroute to the diploma, under terms they find suitable. In all but the highest 
curricular tracks, students and teachers cut a mutually accommodating bargain (Powell, et al., 1986; 
Sedlak et al, 1986; Sizer, 1985). According to terms of the bargain, if teachers let students off with 
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only modest demands on their time and effort during the school day, most students cooperate and 
comply with the rules for classroom conduct. Ironically, for both teachers and students, such an 
arrangement is unfulfiiling. When teachers find little challenge in the classroom, they put their 
energies into other interests -- for example, taking on a second job. Unchallenged U.S. students learn 
less and achieve at lower levels than their counterparts worldwide (Conley, 1993). Most disturbing, 
the flaccidity of U.S. students' intellectual experience in school thwarts their personal development 
(Czikszentmihalyi & Larson, 1984) and, ultimately, impoverishes society. If student engagement with 
academic work is to increase, schools and teachers will have to press for engagement with learning, 
especially for less academically proficient students (Sizer, 1985; Powell, Farrar, and Cohen; 1986). 

School restructuring and stxident engagement. Because the school restructuring movement 
aims at remedying many of the problems besetting U.S. education, the organizational and 
programmatic innovations of restructuring schools may enhance student engagement. Some 
restructuring schools have begun to move away from bureaucratic organization and toward 
organizational forms that are more communitarian and personal, such as multi-year advisory groups 
and the school-within-a-school (Conley, 1993). Instruction as the transmission of information is, in 
some restructuring schools, giving way to an emphasis on the active involvement of students in 
constructing knowledge (Leinhardt, 1992; Newmann & Wehlage, 1993). Since the investment of 
parental interest and energy in their children's education bears out in academic performance (Muller, 
1993; Stevenson & Stigler, 1992), many restructuring schools are making systematic efforts to 
involve parents with student learning as well as in school activities and governance (Newmmann, 
1991b). Because the expectations schools and teachers hold for all students' best performance directly 
influence how well students achieve, restructuring schools often strive to strengthen their commitment 
to equity. The resulting "success for all" instnictional emphases may bring about greater student 
engagement. 
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Previous Research 

Quantitative researchers have focused on student engagement in schools and classrooms as a 
valued outcome in its own right, as well as a predictor of achievement. Employing engagement as an 
outcome, the school *ocused stream of the research has operationalized the construct globally and 
along two dimensions (e.g., Lee & Smith, 1993, 1994). The first dimension is positive and 
attitudinal, students' favorable disposition toward the academic work of school. The second 
dimension is negative and behavioral, students' evident alienation from school, characterized by at- 
risk activity. The classroom-focused stream of the research on engagement has usually investigated 
student involvement with academic work, operationalizing the construct more specifically and for 
panicular settings, for example, as observed on- or off-task behavior in mathematics and social 
studies classrooms (Stodolsky, 1988). 

School restructuring. Since increased student engagement is an expected consequence of 
school restructuring, researchers in the school-oriented stream of engagement studies have 
investigated this relationship. As noted earlier, these researchers have produced evidence of a link. 
Focusing on restructuring in middle schools, Lee and Smith (1993) found higher levels of academic 
engagement among students in schools that had undertaken more extensive restructuring (e.g., 
heterogeneous grouping, flexible time scheduling, schools-within-schools, team teaching, stability in 
homeroom arrangements). At-risk behavior proved independent of the amount of school 
restructuring, however. Investigating the influence of school restructuring in high schools, Lee and 
Smith (1994) found the least engaged students in schools that had undenaken no restructuring. 
Students proved to be more engaged in high schools with a greater academic emphasis as measured by 
aggregated coursework of students in mathematics and science and less variability in academic course- 
taking. 
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Instruction. Researchers focusing on classrooms have examined whether engagement varies 
as a function of instruction, classroom activities, and the subject matter of the class. Although levels 
of engagement did not vary between the elementary school social studies and mathematics classes 
studied by Stodolsky (1988), students in both subject areas were the most engaged by cognitively 
involving instruction. WTiile Nystrand and Gamoran (1991) found high levels of procedural 
engagement in 8th grade English classes, they found little substantive engagement. Substantive 
engagement tended to occur when teachers posed open-ended questions, incorporated student 
responses into further questions, and built discussion around the ideas the exchange generated. 
Discourse quality in 9th grade classes (Gamoran and Nystrand, 1992), however, evidenced no 
consistent relationship with student engagement. 

Achievement. Students achieve at higher levels when they are engaged. Reviews of the 
literature on engagement have consistently documented positive relationships between student 
academic performance and student involvement/participation in the classroom ~ whether elementary, 
middle, or high school (Finn, 1989; Finn & Cox, 1992).^ Supporting previous research with 
evidence from national data, Finn (1993) found that engaged 8th grade students achieved at higher 
levels than their less engaged peers. Examining the literature achievement of 8th grade students, 
Nystrand and Gamoran (1991) found disengagement adversely affected achievement, while substantive 
engagement influenced achievement positively. Among 9th grade students in English classes, 
Gamoran and Nystrand (1992) again found that disengagement negatively affected achievement. 
While behavioral engagement (e.g., completing reading and writing assignments) had a positive 
relationship with achievement, students' reported engagement (e.g., trying hard in class, concentrating 
on the lesson) did not significantly influence achievement. 




Expanding on earlier work (Newmann 1981), Newmann, Wehlage and Lambom (1992) 
articulated a theory of student academic engagement that drew upon the sociological theory of Merton 
(1968) and the psychological theory of Connell (1989). Defining engagement as "students' 
psychological investment in and effort directed toward learning, understanding, or mastering the 
knowledge, skills, or crafts that academic work is intended to promote" (p. 12), the authors proposed 
three bases for student academic engagement: (1) the fundamental human need to develop and express 
competence, (2) school membership, and (3) authentic academic work. Because the need for 
competence is generally inherent, most students begin their school careers motivated to learn. For 
many students, however, their experience of school dulls that motivation or even suppresses it 
entirely. Students are more likely to direct their need to develop and express competence toward 
academic commitment if they develop a sense of membership in the school community (i.e., identify 
with its clearly expressed purpose, experience fairness, caring, success, support) and if the 
scboolwork they are asked to do is authentic (Newmann, Wehlage & Lambom, 1992). 

Finr (1989, 1993) has also proposed a model for student engagement. Broader than the 
definition offered by Newmann et a'l., engagement, according to Finn's conceputalization, is student 
involvement wi*h school. Engagement may take an affective or behavioral form. Affectively, 
engagement implies a sense of belonging and an acceptance of the goals of schooling. Behaviorally, 
engagement is a continuum of developing participation (i.e., compliance with school and classroom 
procedures, taking initiative in the classroom, becoming involved in school activities and, ultimately, 
taking part in school governance). 

Differing from the model Newmann et al. (1992) proposed (engagement as a function of the 
human need to develop and express competence, school membership, and authentic work), Finn 
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situates engagement within an ongoing cyclical process. Participation leads to academic success, 
which, in turn, influences identification with school (i.e., the affective dimension of engagement - 
valuing, belonging). Identification increases the likelihood of future engagement. The depiction of 
engagement as a product of cumulative school experience is a strength of Finn's model. By 
portraying engagement as largely a function of the individual, however, the model largely omits 
influences from organizational environments, except for instruction. 

Building primarily upon the model proposed by Newmann, Wehlage &. Lambom (1992), the 
model of student engagement proposed and evaluated here directly incorporates two of their bases: 
students' need for conq)etence as a motive for engagement and the importance of authentic work as a 
means of enlisting students' intellectual involvement. School membership, the third base posited by 
Newmann, et ai., is reconfigured in the present model. A related construct, structures of support for 
learning, incorporates membership in school as "experience of school as a learning community." 
This construct also includes two other salient structures of support for learning - classroom support 
(including the teacher's high expectations for student performance and peers' cooperation in the 
learning process) and parental support (including parents' interest, discussion, and involvement with 
the child concerning school and schooling). Students' past experience of school (success or 
alienation), a component of school membership in the Newmann et al. model, is represented 
separately here and hypothesized to influence present engagement in the academic work in 
mathematics or social studies classrooms (cf Finn, 1989, 1993). 

Theoretical Model 

The theoretical model guiding this study is displayed in Figure 1 . Read from left to right, the 
model focuses on the attributes of students and classrooms hypothesized to explain variation in student 
engagement in academic work. Each of these student and classroom characteristics is hypothesized to 
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have a direct relationship with engagement. Listed in the first block are student experiences of an 
academic environment - including authentic academic work and structures of support for learning 
(i.e., school, classroom, and family); students' basic orientation toward school (success vs alienation); 
and personal and academic background (i.e., gender, self-concept [a proxy for the need to develop 
competence], race-ethnicity, social class, and prior achievement). Listed in the second block are the . 
characteristics of classrooms hypothesized to influence the average engagement level of students. 
Classroom contextual features include the average level of authentic work, average level of prior 
achievement, and subject area (mathematics vs social studies). Teachers' pedagogical approach, i.e., 
the choice of instructional methods, also has the potential to influence student engagement. The 
model categorizes pedagogy as progressive and traditional. 

Insert Figure 1 about here 

Variation in Student Engagement Within Classrooms 

In a given classroom, why are some students more or less engaged than others? Even though 
stude Its in the same classroom are (presumably) exposed to the same subject content, teacher, 
instruction, and classroom peer group influences, some of them will be attentive, actively involved 
with the subject matter, sometimes initiating questions, while otliers are passive, distracted, even 
disruptive. To explain within-classroom variation in engagement, this investigation hypothesizes the 
importance of four facets of student experience: orientation toward school; authentic academic work; 
structures of social support for learning; and personal background. 

Orientation toward school. Schooling is a powerful societal institution incorporating most 
young Americans until young adulthood. Schools attempt to impart a basic education (i.e, socially 
meaningful knowledge), to socialize students into democratic citizenship, and to make available the 
skills and credentials needed for productive and rewarding social roles. Students who are goal- or 
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performance-oriented are likely to accept the purpose of schooling and to invest in its rituals and 
requirements. Usually, academic success rewards such application. Because their less purposeful 
peers find the activities of school alien to their interests and, therefore, meaningless, they are likely to 
resist academic involvement. Not only are they less academically successful than students who 
identify with the goals of schooling, they may break rules, skip school, and otherwise rebel. Because 
students enter classrooms more or less integrated with the goals of schooling, their predispositions to 
become academically engaged differ dramatically. 

Authentic work. Despite the commonalities in the instructional experience of students in the 
same classroom, students may differ in the extent to which they view their school work as authentic. 
Students who volunteer or who are called on more often, for example, are more likely to experience 
meaningful intellectual interactions. Differences in students' personal and experiential backgrounds 
might also affect their perceptions of schoolwork as authentic. The model proposed here assumes that 
students' perceptions of authentic work reflect their experience of authentic instruction (Newmann & 
Wehlage, 1993). 

Authentic instruction departs from the stereotype of traditional instruction ~ the lecturing 
teacher who imparts knowledge and the passive student who absorbs the :nformation and who must be 
prepared co restate it in recitations or tests and quizzes. Authentic instruction emphasizes cognitively 
active students who construct meaning and produce knowledge, an instructional process that attains 
higher order objectives through disciplined inquiry, and student work that has value and relevance 
beyond the classroom. When instruction is characterized by these standards, students are likely to be 
experiencing authentic academic work. 

Structures of social support for learning. The extent of support for learning that students 
receive from the groups to which they belong influences engagement in the classroom (Brown, 1993; 
Cusick, 1991; Lamborn et al., 1992). Important structures of academic support available to students 
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include fhe school itself as a social environment that promotes learning; teachers and peers in the 
classroom who hold high expectations for each other academically and who are actively involved with 
each other in learning; and demonstrated parental interest in their children's academic work and 
school-related activities. To the extent that these support structures are developed and focused on 
learning, students' engagement in thj classroom will be greater. - 

Students' self-concept. Self-concept is a proxy for the need to develop and express 
competence. A social-psychological construct, self-concept is represented by several dimensions ~ 
among them self-esteem, "based on a sense of competence, power, or efficacy" (Gecas, 1982, p. 5). 
As related to competence, self-esteem derives from the ability to perform effectively, as demonstrated 
in the past and expected in the future (Connell, 1990). The self-concept thus provides a motive for 
personal actualization or the desire to achieve competence (Gecas, 1982). Without a positive self- 
concept, furthermore, powerlessness and, ultimately, alienation result. 

Students' social background. Theoretically, the link between social background and 
engagement derives from Merton's theory of alienation (1968). When cultural goals and the 
institutional means to attain them are incongnient for some groups ~ for example, racial and ethnic 
minorities ~ the disjuncture may result in their alienation. Students desirous of achieving a respected 
place in society, but who do not regard their schools as preparing them for that status may thus 
becdme alienated. Although school ought to function as an opportunity structure for young people, 
knowledge that the structure fails to provide them opportunity could result in disengagement among 
some students. 

Generally consistent with Merton's theory, most previous research has documented 
engagement as varying according to the social and academic background of students. As 
socioeconomic status increased, for example, so too did the level of student engagement (Finn, 1991, 
1992; Lee & Smith, 1993, 1994). Similarly, prior achievement and academic background positively 
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influenced academic engagement and negatively influenced at-risk behavior (Lee & Smith, 1993, 
1994). At all grade levels - elementary, middle, and high school -- student gender differentially 
affected engagement. The relationship between gender and engagement seems to depart from 
Merton's theory. Since the gender stratification in society favors males (Epstein, 1988), they would 
seem likely to find society's institutions, such as schools, in line with their goals. Boys would, 
accordingly, seem more likely to be engaged with school. Girls, however, have consistently proven 
more academically engaged than boys (Finn, 1991, 1992; Lee & Smith, 1993, 1994). 

The relationship between minority status and smdent engagement documented by previous 
research is not clearcut. Minority elementary school students proved less involved academically in 
Finn's (1992) study. Minority middle school smdents were more inclined to at-risk behavior than 
their non-minority counterparts, but the groups did not differ on academic involvement (Lee & Smith, 
1993). However, minority high school students (in an analysis controlling for engagement during 8th 
grade) proved more likely to be positively oriented toward academics than Asian and non-Hispanic 
white students (Lee & Smith, 1994). 

Variation in Student Engagement among Classrooms 

Why does engagement vary from classroom to classroom? Reasonably, between-classroom 
differences in academic application may be a function of the subject matter, the type and quality of 
instruction, and the energy students derive from the classroom intellectual context, that is, the 
achievement level of the class and the average level of authentic work. 

Subject matter. Subject matter conditions the professional practice of teachers (Stodolsky and 
Grossman, 1992); that is, teachers' views of knowledge, their instructional approaches, and their 
goals for students. These dimension of practice clearly have implications for student engagement. As 
a "basic" subject, math differs from social studies (an "enrichment" subject), in that its content is 
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sequential and predictable (Stodolsky, 1988). Students consider themselves more teacher-dependent in 
mathematics, where the teacher is the "source" of knowledge, compared to social studies, where the 
teacher is the "elaborator" of knowledge (Stodolsky, Salk & Glaessner, 1991). Among students at all 
grade levels, social studies is one of the least liked subjects - considered "easy" b^ high school 
students and "difficult" by elementary school students (Goodlad, 1984; Stodolsky et al.). Social 
studies' inherent link to life outside the classroom might generate more interest among students than 
mathematics does because of its relevance to their experience. The public evaluation students expect 
in their mathematics classes, however, might encourage them to greater academic application there 
than in social studies classes. 

Instructional methods. Teachers who use methods typically described as progressive, 
according to common perception, are more likely to succeed in engaging students than those who use 
traditional methods (Goodlad, 1984). Progressive methods include such approaches as active 
learning, cooperative groups, hands-on experiences, and independent research. Because these 
methods are student-centered and oriented to involving students directly in the construction of their 
learning in ways directly counter to the transmission model, progressive teaching may succeed in 
engaging students. Because traditional methods are teacher- :entered, usually involving some variant 
of lecture and discussion, they niay result in student disengagement. Ultimately, however, the 
relative efficacy of progressive vs traditional methods in enhancing student engagement is an 
empirical question. Since instructional approaches do not assure authentic student work (Newmann, 
Wehlage & Secada, 1995), whether teachers use progressive or traditional methods may not affect 
student engagement directly. 

Classroom context. While authentic work is a dimension of student academic experience in 
the within-classroom model, in aggregate, authentic work also functions as a normative characteristic 
differentiating classrooms. That is, classrooms in which the average level of authentic work is high 
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are, reasonably, especially engaging environments. Although the classrooms studied in this 
investigation are generally heterogeneously grouped, even due to random variation, the achievement 
level of some classes would be higher than others. While the higher levels of average achievement 
would not in itself suggest higher levels of student engagement, prior research has supplied evidence 
of such a link (e.g., Lee & Smith, 1993, 1994). 

Method 

Sample 

A major focus of the OERI Center for the Organization and Restructuring of Schools is the 
experience of students, including the relationship of instruction and school organizational features to 
student engagement in academic work. The sample of restructuring schools studied by the center 
consists of 24 public schools (8 elementary, 8 middle, and 8 high schools) selected through a national 
search (Berends & King, 1994). Important criteria for a school's inclusion in the study were 
indications of being well along in the process of restructuring boih student experiences and the 
professional life of teachers (Newmann, 1991b).'' The focal population for this study is a portion of 
the school restructuring sample, students (and their teachers) in grades 5, 8, and 10, from six core 
classrooms (3 mathematics and 3 social studies) in each of 24 re^>tructuring schools.^ Participating 
schools selected the core classrooms in each subject area according to two criteria specified by the 
Center: "At least one core teacher in each subject area is involved iii the school's effort to restructure 
student experiences and ... the three classes reflect the range of student achievement within the grade 
as a whole" (Center for the Organization and Restructuring of Schools, 1992). 
Data 

The data for this investigation, collected during 1991-1994, are primarily the survey responses 
of social studies and mathematics teachers and their students in the targeted core classrooms. In 
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addition many ether questionnaire items, teachers ~ with their target class in mind - responded to 
questions about thsir instructional goals and practice, the extent of their control over the curriculum 
and classroom, and their personal and professional backgrounds. Students responded to survey items 
about their attitudes, behaviors, and experiences in either a target mathematics or social studies class; 
about their experience in school more -generally ; and about their personal and family background.^ 

The survey response rates were quite good, and the representation of students by grade level 
and subject area is fairly well-balanced. More than 3,660 students in 143 of the 149 core classrooms 
completed surveys, averaging 26 students per class.'' Classes with survey data from both teachers and 
students numbered 138. The item resporise rates for completed teacher surveys was 95 percent and 
for completed student surveys, 96 percent. Table 1 displays the analytic sample for this investigation. 
Mathematics and social studies classrooms are equally represented at all grade levels. Except for the 
elementary school level where the number of students in social studies is less than in mathematics, 
social studies classes, on average, enroll more students (1904 vs 1765). Overall, the elementary 
students outnumber their counterparts in middle and high school by about 175 students al each level. 

Insert Table 1 about here 



Restructuring Schools and the Universe of U.S. Public Schools 

Compared with the universe of U.S. public schools on a set of such typical statistics as size, 
social class, and minority enrollment, the restructuring schools in the sample are larger - averaging 
777 students vs a national average of 522. The schools in the restructuring sample enroll more Black 
and Hispanic students ~ (20.6 percent Black students vs 16.3 percent nationally) and (21.7 percent 
Hispanic students vs 11.8 percent nationally). A substantial portion of the sample is poor - 
averaging 37 percent of students on free or reduced lunch ~ but less than the percentage nationally, 
56%.« 
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Disaggregated by grade level, some restructuring school differences with the comparable level 
in the universe of public schools are especially pronounced. The restructuring elementary schools, 
for example, enroll three times as many Hispanic students as other U.S public elementary schools (41 
percent vs 13.5 percent). The restructuring high schools eruroll more than twice as many black 
students as other U.S. public high schools (34 percent vs 15 percent).' Unlike the elementary and 
high schools in tho restructuring sample, which enroll more minority students than their national 
counterparts, the restructuring middle schools enroll fewer — 5 percent fewer Blacks and 2 percent 
fewer Hispanics. (Appendix A contains a complete comparative profile on these statistics for all three 
grade levels.) 

Because the Center collected prior ability measures on the students in the core classes — 
baseline tests using selectee! items from the NAEP reading and mathematics test and a center-designed 
writing test scored using the is'AEP system — student ability may also be compared to a national 
sample. These coinparisons also vary by grade level. The elementary schools are at the national 
average in mathematics and slightiy above in reading.'" The middle schools rank well above the 
national average, especially in matl .ematics. The high schools fall considerably below the national 
average in mathematics and, even more, in reading. The high schools', low scores may be 
attributable, at le<: Tiartially, to the discrepancy between the normed grade level for the high school 
test items (Grade 12) and the grade level of the students taking the test (Grade 10). (Appendix B 
contains the comparative profile on achievement for the restructuring schools against national norms 
for all three grade levels.) 

Measures 

Dependent variables Student engagement in academic work is constructed as a factor 
containing four component measures: student effort (In social studies/ mathematics class, hov,' often 



17 



do you try as hard as you can?); attentiveness (How often do you pay attention in this class?); lack of 
boredom in class (Often I feel bored in this class, [reversed]); and completing class assignments 
(About how often do you complete your assignments for this class?). The internal consistency of this 
measure (Cronbach's alpha) is .69. Engagement as reported by students is substantially correlated 
with the level of engagement Center researchers observed in the classrooms of these same students, 
r=.374." By providing a complementary objective assessment of classroom engagement levels, the 
observers' report validates the more subjective student rating. (Appendix C contains additional details 
on the dependent measure and all other variables incorporated in the investigation). 

Independent variables measured on students. Orientation toward school is represented by two 
measures ~ academic success and alienation. Academic success is operationalized by student grade 
point average in English, science, mathematics, and social studies, a proxy reflecting student 
integration with the goals of schoc'irig. For elementary school students, alienation is measured as the 
mean of two standardized items: the frequency (during the year of the survey) with which a student 
was late for school and got into trouble for not following school rules. For middle and high school 
students, the measure of alienation also includes the frequency with which the student cut or skipped 
classes, was put on in-school suspension, or was put on probation or suspended from school. 

Authentic work, constructed as a factor, consists of four component measures relating to the 
frequency with which the student is involved in meaningful academic experiences in the core 
mathematics or social studies class: (a) You are asked interesting questions and solve new problems; 
(b) You dig deeply into understanding a single topic; (c) You apply the subject to problems and 
situations in life outside of school; (d) You discuss your ideas about the subject with the teacher or 
students.''' 

Structures of support for learning is operationalized by three measures: (1) School support 
for learning, the mean of its standardized components, incorporates five items tapping students' 

18 



20 



school experience - whether the student is put down by other students (reversed), does not feel safe 
at school (reversed), finds that disruptions by other students get in the way of his or her learning 
(reversed), believes most teachers really listen to what she or he has to say, and thinks that she or he 
(and friends) are treated fairly. (2) Classroom support, a factor, reflects a combination of high 
expectations for achievement and help for learning from teachers and peers. (3) Parental support is 
constructed as an index. For elementary students, the index is the mean of two factors ~ parental 
involvement at school and the frequency of discussion among students and their parents on academic 
matters. For secondary school students, the index includes a third factor ~ the frequency with which 
students discuss their school program with their parents and, as well, their college plans, including 
taking the SAT or ACT examinations. 

Self-concept meast.'.res students' attitudes about themselves and their competence from a series 
of personally evaluative items. The elementary students' measure comprises 5 items: I feel good 
about myself; I am a person of v/orth, the equal of other people; I am able to do things as well as 
most other people; at times, I think I am no good at all (reversed); and I feel I do not have much to 
be proud of (reversed). The secondary students' measure of self-concept includes two additional 
items: On the whole, I am satisfied with myself; and I feel useless at times (reversed). 

Demographic measures include: gender (female), a dummy variable, coded (l = Female) 
(0 = Male); race (Black) and ethnicity (Hispanic), each an effects-coded variable, (l=Black [or 
Hispanic]) (0 = Hispanic [or Black]) (-1 =White); and socioeconomic status. For elementary students, 
SES is the mean of household items and household features (summed and standardized). For 
secondary students, SES is the mean of household items and household features and the mean of 
parental education (summed and standardized). Prior achievement was measured early in the survey 
year as the student's score on a standardized test using NAEP items. Students in the mathematics 
core classes took a mathematics test; students in social studies core classes took a reading test and 
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submitted a writing sample, scored using a Ni*.£P rubric. The social studies achievement score is the 
mean of the reading and writing test scores. 

independent variables measured on teachers and classrooms. Of the three independent 
variables typifying classroom context, two are aggregated from student data - average prior 
achievement and the average level of authentic work. Subject area is a dummy variable, coded 
(l=Math) (0= Social Studies). Pedagogical approach, constructed from the responses of teachers to 
survey itemis, contrasts traditional methods (lecture, frequent tests and quizzes, drill and practice on 
the basics) with methods typically regarded as more progressive (students working in small groups; 
discussions; use of lab periods and hands-on activities; out-of-school activities, such as field trips and 
community experiences; individual students receiving teacher help; problem-solving or research 
requiring the organization and integration of knowledgs). 

Analytic Approach 

The initial set of analyses examines observed mean differences on the modeled constructs by 
subject area and grade level. The examination employs two-way analysis of variance (ANOVA), 
which permits the simultaneous evaluation of the main effects (subject and level) and the possible 
interaction of these effects (subject-by-level). If differences occur in the relationships among the 
modeled constructs by grade level and subject, that interaction would have implications for the 
multivariate analyses to follow. Rather than investigate the theoretical model on the combined sample 
(i.e., for students in elementary, middle, and high schools simultaneously), it would be necessary to 
conduct the multivariate analyses separately by grade level, while controling for subject area. 

Because the multivariate analyses involve nested data, students in classrooms, the principal 
analytic technique is multilevel, hierarchical linear modeling (HLM) (Bryk & Raudenbusch, 1992). 
HLM partitions the variance in th^ dependent variable into its within- and between-group components. 
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in this instance, classrooms. The HLM analysis, accordingly, employs two equations: (a) a within- 
group model that explains variation in the outcome for each classroom as a function of individual 
characteristics and random error, and (b) a between-group model that explains tlie variation in the 
outcome as a function of the differing characteristics of classrooms and a more complex error term 
(Bryk & Raudenbusch). Appendix D contains technical defails on the HLM models employed in this 
investigation. 

An unconditional HLM model (i.e., a model with no predictors at either the within-group or 
between-group levels) serves to estimate how much variability in student engagement exists within- 
and between-classrooms and is potentially explainable by the analyses. Liilike much research 
ernploying HLM primarily to explain between-group differences, this investigation is equally aimed at 
explaining wnhin-group variation in engagement. The within-classroom HLM model provides 
parameter estimates of the structural relationships hypothesized to influence engagement, controling 
for personal background differences.'^ The between-classroom model, adjusted for within-classroom 
differences and for random between-classroom effects, investigates why average student engagement 
is higher in some classrooms than in others. 

Results 



Descriptive Analyses 

Grade level differences. Engagement in academic work, unadjusted for any other influences, 
declines as grade level increases (Table 2). While elementary students report being most engaged and 
high school students least, middle school students rank between them, less involved than students in 
elementary school, but more involved than their high school counterparts. Overall, students in 
mathematics classes report greater engagement than their peers in social studies, but the presence of a 
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significant subject-by-level interaction indicates that the subject area effect varies significantly by 
grade level. Figure 2 displays the interaction in deviation score metric. While students in all three 
grade levels rank almost the same in mathematics engagement, only middle school students rank about 
as high in social studies engagement. Engagement in social smdies is relatively very low for smdents 
in elementary and high schools. 

Insert Table 2 about here 

Measured by their comparatively low level of academic success (GPA) and their 
comparatively high level of alienation, high school smdents report the least positive orientation toward 
school. Among students in the core class subject areas, differences in the levels of tardiness and rule- 
breaking are not significant. Cutting class and being suspended is more coirirnon among the social 
studies students. Although authentic academic work does not vary by grade level, it does vary by 
subject area. Mathematics students report higher levels of authentic work than their peers in social 
studies. 

Insert Figure 2 about here 



Support structures for learning tend to vary both by grade level and subject area. While 
perceptions of their school as a positive learning environment do not var>' among smdents by grade 
level, they do vary by subject area. Students in mathematics classes are more likely to regard their 
school environments as supportive. Classroom support, on average, is greatest among elementary 
school students and among mathematics students. The presence of a significant subject-by-level 
inteiHCtion indicates that the mathematics effect is not consistent over grade levels. In fact, 
elementary school students report greater levels of classroom support for learning in social studies 
than in mathematics, while middle and high school students report greater support for learning in 
mathematics. Elementary school students also experience greater parental interest in and involvement 
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with their schooling than do middle and high school students. Among the secondary school students, 
those in high school report more conversations with their parents about their school piograms and 
college preparation plans than students in middle schools do. 

The proportions of girls at all school levels and in core subject classes is statistically 
equivalent to the proportions of boys. Considered by racial or ethnic background, however, the 
distribution of students varies among these groups. The proportion of Black students is highest in 
high schools and greater in mathematics than in social studies classes. Hispanic students are most 
numerous at the elementary level, but equally likely to be in mathematics as in social studies classes. 

Social class, measured by household items and other household features, is highest for middle 
school students. A significant grade level by subject interaction is present, however. While 
elementary and high school mathematics students rank above their classmates in social studies on 
social class, among middle school students the SES of social studies students exceeds that of their 
peers in mathematics class. Middle school students' parents have attained higher levels of education 
(almost 15 years of formal schooling compared to just over 13.5 years for high school students' 
parents). Parents of students in social studies have averaged a few months more of formal education 
than have mathematics class parents' (14.4 vs 14.2 years of schooling). Elementary school students 
are most positive in self-concept, a characteristic that does not vary by subject area. 

Prior achievement on the standardized baseline test is highest for elementary school students 
and, generally, for students in social studies. An exception exists, however, as the interaction 
indicates. Compared to middle and high school students, for whom the reverse is true, elementary 
school students rank higher on baseline achievement in mathematics than they do in social studies. 

Classroom characteristics differ somewhat for students by level and subject (Table 3). The 
average prior achievement level of students is highest for elementary school classrooms and higher for 
social studies, except at the elementary school level v/here mathematics students outscored their 
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counterparts in social studies. In both elementary and middle school classrooms, the average level of 
authentic student work is higher than in high school classrooms. Overall, the level of authentic work 
reported by students in mathematics classes tops that of social studies. At all three grade levels, 
teachers use progressive and traditional teaching methods with comparable frequency. Progressive 
methods are somewhat more prevalent in social studies classes, however, and traditional methods 
slightly more common in mathematics classes. 



Insert table 3 about here 



Multivariate ELM Analyses 

Because of the subject-by-level interaction explained above on the dependent variable, the 
multivariate HLM analyses will be conducted separately by grade level. While statistical 
considerations necessitate the strategy, conducting separate analyses by grade level permits the testing 
of the theoretical model for engagement at three different levels of schooling. If the patterns prove 
similar for students at all three grade levels, the model can be considered generally appropriate and 
educationally applicable across grade levels. 

The first of the HLM analyses employs an unconditional model to provide estimates on the 
psychometric properties of the dependent measure. These estimates include the amount of variance 
partitioned within- and between-classrooms and the reliability of the outcome ;'S an estimate of the 
true classroom mean. At all tbjree grade levels, most of the variance in engagement occurs within 
classrooms (Table 4) - a high of 86% at the high school level, a low of 80% at the middle school 
level. Thus, engagement is largely a function of individual student characteristics and experiences. 
Between classroom variance is greatest among the elementary school classrooms (17%), least among 
high school classrooms (9%). The HLM estimate of the reliability of the depeident variable. 
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reflecting both variation in the means across classrooms and the sample size, is highest at the 
elementary level (.83) and lowest at the high school level (.67). 

The intraclass correlation represents an adjusted estimate of the between-classroom variability 
for engagement among the grade levels that takes into account both the amount of within-classroom 
variability and the internal consistency of the dependent variable.'** With these adjusi.Tients, the 
amount of between-classroom variability is somewhat larger than the estimates provided above -- 26% 
for elementary schools, 19% for middle schools, and 13% for high schools. The theoretical model 
proposed earlier will attempt to explain the variability in engagement that exists within- and between- 
classrooms. 



Insert table 4 about here 



Within-classroom HLM model. The theoretical model exhibits a strikingly similar pattern of 
relationships for engagement in academic work at all three grade levels. Table 5 displays these 
results in effect size metric.'^ At all three grade levels, students' basic orientation towa.-'d school 
strongly influences their academic application in the classroom. The salience of previous success is 
greatest for high school students (ES = .78, P <, .001). Although the relationship is still very strong, 
previous success influences engagement comparatively less for middle school students (ES = .56, P 
< .001). Elementary school students rank between students at the other two levels in the importance 
of previous success to engagement (ES = .61, P <. .001). Alienation toward schooling seriously 
detracts from classroom engagement for students at all grade levels. Alienation from school is 
especially detrimental to classroom engagement for middle school students (ES = -.81, P < .001) 
and high school students (ES = -.71, P < .001), less so among elementary school students (ES = - 
.50, P < .001). 
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For students at all three grade levels, authentic academic work is the most powerful 
contributor to engagement. For middle school students, the effect of authentic work is most striking 
(ES = .96, P < .001) and somewhat less so for high school students (ES = .82, P < .001). 
Although the engagement of students at all levels benefits from support structures for learning, high 
school students profit most. The total effect of support structures for learning (school, classroom, 
and parental support sunmied) is greatest for high school students (ES = 1 .40), followed by middle 
school (ES = 1.01) and elementary school students (ES = .95). 

For elementary school students, of all the support structures, regarding school as supportive 
to learning has the strongest relationship to classroom engagement (ES = .39, P < .001). Parental 
support is the least important (ES = .22, P < .001). For middle and high school students, parental 
support is a slightly more important influence on engagement than is the learning environment of the 
students' school. Most important of these structures for high and middle school students, however, is 
support for learning within the classroom (ES = .70) and (ES = .49), respectively. For elementary 
students, classroom sup-,>. rt (ES = .34, P < .001) influences engagement somewhat less than the 
school environment, but more strongly than parental support. 

At all three grade levels, but especially among middle and high school students, female 
gender increases the likelihood of classroom engagement (for middle and high school students, ES = 
.48 and .47, respectively, P < .05.; for elementary students, ES = .38, P < .01). A positive self- 
concept most enhances engagement for middle school students (ES = .39, P ^ .01), less so for 
elementary and high school students (ES = .24, P < .01, and ES = .22, P < .05, repectively). 

The theoretical model explains about thirty percent of the within-classroom variation in 
engagement for students at all three grade levels ~ less for elementary and middle school students 
(29%), more for high school students (32%). 
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Insert table 5 about here 



Between-classroom HLM model. The between-classroom HLM model adjusts for the 
modeled within-classroom differences. Unlike the student-level (within classroom) relationships 
discussed above which were similar for students ai all three grade levels, classroom effects on 
engagemfci^t differ by grade level and subject (Table 6). For elementary and high school students, 
mathematics classrooms produce much higher levels of engagement (ES = 1.00 and .92, respectively, 
P < .01), but for middle school students engagement is weaker in mathematics classrooms (ES = - 
.70, P < .05). 



Insert table 6 about here 

The average prior achievement level of the class proved to have no relationship with student 
engagement, thus that contextual feature of classrooms does not appear in the final model. Authentic 
work as a feature of classroom context is important, however, in the upper grades, especially in high 
school, as an influence on average student engagement. Among elementary schools the level of 
authentic work reported by students in the classroom does not influence average levels of classroom 
engagement. In middle and high schools, however, this contextual feature strongly differentiates 
engagement levels among classrooms (ES = .46, P < .05, and ES = .66, P < .001, respectively). 

In high school classrooms, once the subject area and average level of authentic work is taken 
into account, engagement is independent of the pedagogical approach of teachers. Wliether the 
teacher typically employs progressive or traditional teaching methods, given these adjustments, is 
irrelevant. Progressive methods do not significantly enhance the engagement of middle school 
students, but traditional methods do (ES = .53, P < .05). Among elementary school students, 
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progressive pedagogy is associated with lower levels of classroom engagement (ES = -.40, P < .05), 
and traditional pedagogy with higher levels (ES = .53, P < .05). 

The findings on the relationship of pedagogy to student engagement ~ particularly the positive 
relationship of traditional methods in elementary and middle school classrooms and the negative 
relationship of progressive peda^^aogy in elementary schools -- are somewhat surprising. An item 
analysis of the relationship to engagement of each component in the progressive and traditional 
pedagogy constructs is instructive in accounting for these findings. (See Appendix C for a list of 
items comprising the traditional and progressive pedagogical constructs.) For elementary school 
students, the traditional approach of frequent tests and quizzes is positively associated with higher 
levels of engagement (.22), while teachers' working with individual students is negatively associated 
(-.13) with average classroom involvement. Among mdddle school students, two components of 
traditional pedagogy have a positive relationship with engagement ~ frequent tests and quizzes (.26) 
and an emphasis on basic skills and procedures (.28). 



Discussion 



Summary of the Findings 

Confirming the aptness of the theoretical constructs hypothesized to influence student 
engagement in academic work, the investigation supported students' integration with the goals of 
schooling as salient to classroom involvement. Academic success made engagement more likely, 
while alienation from school, defined as at-risk behavior, seriously reduced the likelihood of 
classroom engagement. Authentic academic work exerted a powerful influence on engagement. 
When students experienced a well-developed support system for learning - at home, in school, and in 
the classroom ~ they reported substantially higher levels of classroom engagement. Self-concept, 
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representing a motive to develop and express competence, proved a moderately important influence 
on engagement. Net of these theoretically salient features, girls were considerably more likely than 
boys to report involvement in academic work. 

At the classroom level, mathematics classes engaged elementary and high school students 
much more than social studies, while social studies proved almost as engaging as math for middle 
school students. A classroom context characterized by high levels of authentic work enhances the 
engagement of middle and high school students. When subject area and the level of authentic work 
are taken into account, traditional methods (specifically, frequent tests anci quizzes) account for an 
increase in the engagement of elementary and middle school students. Lower levels of classroom 
engagement are common in elementary schooJs when teachers report an emphasis on giving extra time 
and attention to individual students. 

Notably, social class anc" race-ethnicity did not differentiate the extent to which students 
within classrooms reported engagement in academic work. Since most previous research on student 
engagement documented the infiuence of social background, the absence of such effects may be a 
consequence of the efforts restructuring s':hools are making on behalf of equity. Th( finding that 
girls are more academically involved than boys is consistent with the findings of previous research on 
engagement. Girls' engagement may reflect a greater concern on their part for academic performance 
than boys have (Dweck, 1986), perhaps a result of socialization patterns (Maccoby & Jacklin, 1974) 
or of the differential expectations of teachers (see, for example, Wilkinson & Marrett, 1985). 

The pattern of subject matter, contextual, and pedagogical relationships with engagement 
differs among classrooms by grade level. Mathematics classes rank higher on engagement for 
elementary and high school students, but lower for middle school. Authentic work as a feature of 
classroom context influences the engagement level of middle and high school classes, but is not a 
significant factor in the engagement levels of elementary school classes." An item analysis of 
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pedagogical techniques indicates that classroom engagement for elementary and middle school students 
increases as a fiinction of frequent tests and quizzes. Classes emphasizing basic skills, net of 
discipline and context, have a positive relationship with engagement in middle school classrooms. 

Contribution to Research and Practice 

The focus on student engagement as an outcome of schooling and as antidote to the ultimate 
act ot disengagement, dropping out, has stimulated an interest in engagement theory . Because 
engagement with academic work is fiindamental to students' social development an. I intellectual 
achievement, understanding the structures and processes that influence student engagement is 
indispensable for subsequent research and the formation of policy. By investigating a model of 
engagement entailing within- and between-class room propositions, this study has added a new piece to 
the emerging body of literature on student engagement. From a sociological perspective, this study 
has evaluated and confirmed the importance of authentic academic work and structures of support for 
learning. 

Disciplinary differences. Subject area clearly differentiated classrooms in their average levels 
of student engagement. With adjustments for within-classroom student differences and between- 
classroom contextual differences, mathematics classes outranked social studies in the level of student 
engagement. Because the observers found the levels of authentic instruction equivalent in mathematic 
and social studies, the finding lends support to the influence on engagement of inherent disciplinary 
characteristics. Although current reform in mathematics education has introduced innovative 
curricular and instructional approaches, as a basic subject mathematics is more structured than social 
studies. Despite some variation, content selection is typically sequential and predictable. 
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Because mathematics teachers must prepare students for the next level of instruction by 
ensuring their mastery of present material, moreover, they may press harder for student engagement. 
Mathematics teachers (and students) may also be more vulnerable to the externally imposed pressure 
of standardized testing (Stodolsky et al., 1992). Students' perceived dependency on instruction for 
acquiring mathematics knowledge and the "public" evaluation that tends to typify mathematics more 
than social studies classes may also account for their greater engagement in mathematics classes 
(Stodolsky et al.). Perhaps social studies teachers', in their concern for students* developing positive 
attitudes and values as well as social knowledge, press less for students' engagement in academic 
work than for students' demonstrating good human relations and understanding of cultural differences. 

Progressive vs Traditional Methods. A surprising finding in these analyses concerned 
instructional methods. Net of authentic work and subject matter, progressive methods exerted no 
positive influence on student engagement in academic work, while traditional methods did. As 
suggested earlier, because traditional instruction as operational ized in this study incorporates an 
emphasis on frequent tests and quizzes, academic accountability may motivate student engagement 
when the instruction is traditional. 

The failure of progressive methods in themselves to bring about student engagement suggests 
that when progressive methods are uncoupled from content, they are ineffectual in promoting 
sustained engagement (Newmann, Secada & Wehlage, 1994). Without challenging content and a 
clear focus of inquiry, for example, cooperative groups could be intellectually meaningless and boring 
exercises for students. Independent research could involve students in delving into encyclopedias and 
other reference books to amass facts about a particular subject, but lack a compelling problem or 
question to make the research an intellectually meaningful endeavor. Thus, while suii /ndent- 
centered methods are generally consistent with the direction of authentic student work, student- 
centered methods are not sufficient to ensure students' engagement. 
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Restructuring and academic engagement. A key question for research evaluating the effects of 
restructuring on students' experience of school is whether generalized restructuring efforts are 
sufficient to make a qualitative difference in student outcomes, such as engagement and achievement, 
or whether restructuring requires specific content to bring about specifically sought results 
(Newmann, 1992; Wehlage et al., 1992). Students' commitment to academic effort depends on the 
intellectual substance and quality of pedagogy, according to the latter theory, such as authentic 
instruction and authentic work. The importance of focused, intellectually oriented rationales and 
content not withstanding, Lee & Smith (1993, 1994) found that generalized restructuring ~ 
conceptualized as a movement from bureaucratic to organic school organization and an increased 
emphasis on academics ~ presumably affected the school experience of students in ways that enhanced 
acadeniic engagement. 

Whil'.; not refuting the positive influence of generalized restructuring (organic rather than 
bureaucratic school organization, for example) on student engagement, this investigation provides 
support for the importance of intellectual substance and quality in school restrucuring initiatives. 
Within a sample of nationally selected restructuring schools, chosen because of significant innovation 
in student experience and the professional life of teachers, considerable variation exists in student 
engagement. Specific restructuring content ~ such as authentic academic work and structures of 
support for learning ~ proved important in raising student engagement even where generalized 
restructuring was taking place. 
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Technical Notes 



' !n the 1980s the dropout rate was especially high for minority students. While the dropout rate for 
white students during the decade ranged from 10-13%, for Blacks the rate ranged from 16-24%, and 
for Hispanics, from 28-44% (National Center for Education Statistics, 1993). In some central city 
high schools, over half the students drop out before graduation (Bryk & Thum, 1989). 

^ Investigating the relationship of engagement to the achievement of the students who are the focus of 
this study, the Center on the Organization and Restructuring of schools' preliminary analyses of three 
years of data has also found a positive and significant relationship between student engagement and 
achievement (net of social background and prior achievement). 

^ The hypothesized influence of strucmres of support for learning on student academic engagement is 
coasistent with Bronfenbrenner's (1986) theory of the importance of mesosystems in nurturing human 
development. A mesosystem is a system of microsystems, such as peer group, classroorii, school, 
family, schoolwork. The notion of a mesosystem inplies the connection between two or more 
systems in the person's experiential framework. For students who have a substantial mesosystem 
working for them (i.e., providing support), engagement is likely to be high. To the extent students 
lack a supportive mesosystem, engagement is likely to be low. 

" Criteria also included progress in restructuring of school governance, management, and leadership 
and in the coordination of community resources. 

^ In addition to the teachers and students in the core classes, all teachers in. each school and all 
students in the targeted grade level for each school were surveyed. 

* The elementary student survey is an abbreviated version of the survey administered to secondary 
(middle and high school) students. Most items in the elementary school survey have an identical 
counterpart item on the secondary school survey. In some instances, however, the range of response 
options differs for the tv/o levels. When such differences occur, the item is rescaled - either through 
collapsing the item into common categories or standardizing the item, first on its sample (i.e., 
elementary or secondary), then re-standardized on the entire sample. 

'' Because of a scheduling innovation, one of the restructuring schools in the sample, provided 1 1 
core classes for the study. Since only one of the core classes observed during the fall visit continued 
into the winter trimester (when the majority of the original core students were involved in an 
internship program), 5 new core classes were added (2 social studies, 3 mathematics). Although their 
students and sometimes their class titles changed, the core teachers themselves remained the same for 
both semesters. Thus, while the study involves 144 core teachers, the total number of core classes is 
actually 149. 

* J. Derr, U.S. Department of Agriculture, Food and Nutrition Services, Child Division (personal 
communication, October 12, 1994). 

' The larger over-representation of Hispanic students at the elementary school level and of Black 
students at the high school level is influenced considerably by the almost total minority populations of 
two of the elementary schools and one of the high schools. 
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' Because the center assigned its own topic for student writing and evaluated the writing sample 
using its own norms for scoring, the writing scores of the restructuring schools sample are not 
appropriately compared with the scores of the NAEP sample. 

" Criteria for measuring observed engagement ranged from its lack ~ disruptive disengagement ~ to 
its full expression - all or almost all students seriously involved in the substance of the lesson 
throughout the class. Being on-task, paying attention, doing the assigned work, exhibiting interest, 
taking initiative, interacting cooperatively with others in classroom activities signaled engagement to 
the observers (Center on Organization and Restructuring of Schools, 1992). 

Each of these items corresponds to a standard of ? jthentic instruction on which observers rated the 
core classes. Students' reports of being asked interesting questions and solving new problems 
corresponds to observed higher order thinking; digging deeply into understanding a single topic, to 
observed depth of knowledge; applying the subject to problems and situations in life outside of 
school, to observed connectedness; discussing ideas about the subject with the teacher or other 
students, to observed substantive conversation. 

The correlations between student reports of authentic instructional activity and observed 
authentic instruction are stisticaly significant: for higher order thinking, r=.232; for depth, r=.270; 
for connection to the world beyond the classroom, r=.146; for substantive conversation, r=.272. 
The correlations are computed from student reports of instruction aggregated to the classroom level in 
relation to the average rating of observed instruction over four time points. 

For these analyses the continuous independent variables are centered around the grand mean. All 
of the independent variables are, in HLM terminology, fixed ~ that is, the slopes do not vary 
randomly between classrooms. 

The intraclass correlation is computed as the ratio of the between-classroom variance (tau) to the 
total variance (tau + sigma-squared) with sigma-squared adjusted for measurement error. 

Use of the effect size metric provides a benchmark for answering the question sometimes posed to 
educational researchers: How important (i.e., "big" or "small") is the relationship reflected by the 
coefficient? The effect size metric is expressed in standard deviation units, computed by dividing the 
HLM gamma coefficient (standardized for continuous variables [M=0, SD?=1], otherwise dummy- 
coded [0,1]) by y.he HLM-estimated standard deviation for the appropriate outcome variable. An 
effect size is small if it is less than .1; moderate, if between .2 and .5; large, if over .5 (Cohen, 
1977; Rosenthal & Rosnow, 1984). 

This phenomenon recalls the observation made by Goodlad (1984) that elementary school students 
appeared to be more engaged than students at other grade levels, although nothing about the 
classroom context or pedagogical process seemed to warrant that engagement. 
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Table 1 

Definition of the Analytic Sample 





XVldLnCIUaLlCS 




oociai otuuics 


Combined 






Classrooms 






Elementary 


24 




24 


48 


Middle 


23 




23 


46 


High 


24 




25 


49 


1 ocai 


/I 




72 


143 






Students 






Elementary 


688 




660 


1348 


Middle 


535 




616 


1151 


High 


542 




628 


1170 


Total 


1765 




1904 


3669 
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Table 4 

Student Engagement in Academic Work: HLM Estimates of Psychometric Properties (N = 3,369 students, 
143 classrooms) 







Deoendent Variable 




Engagement in Academic Work 




Elementary 


Middle High 


Within-Classroom Variance 
(Sigma-Squared)* 


.82 


.80 .86 


Between-Classroom Variance 
(Tau) 


.17 


.13 .09 


Reliability 


.83 


.77 .67 


Intraclass Correlation 


.26 


.19 .13 



Parameter estimates are computed with a fully unconditional HLM model. 



The intraclass correlation for engagement in academic work is the amount of between-classroom variance 
explainable by the HLM analyses. The intraclass correlation is computed with the within-classroom 
variance adjusted for the reliability (Cronbach's a) of tlie dependent measure: for elementary students, .60; 
middle school students, .73; high school students, .73. 
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Table 5 

Student Engagement in Academic Work: HLM Within-School Models 







Dependent Variable 








Engagement in Academic Work 


Variables 


Elementary 


Ik if* J J 1 

Middle 


High 


Intercept 

(Classroom Average) 


-.15 


-.18 


-.25 


Orientation toward School 








Success 


.61 *** 


.56 *** 


*** 


Alienation 


-.50 *** 


-.83 *** 


. jl *** 


Authentic Academic Work 


.68 *** 


9g *** 


32 *** 


Support Structures for Learning 








School 


2p *** 






Classroom 


34 *** 


49 *** 


7Q *** 


Parental 


.22 *** 


.27 * 


3g *** 


Personal and Academic Background 








Female 


.38 ** 


.48 * 


.47 * 


Self-Concept 


.24 ** 


39 ** 


.22 * 


% Within-Classroom 
Variance Explained 


29.4 % 


28.7 % 


32.1 % 



* P < .05 ** P < .01 *** P < .OOi 

"The effects displayed in this table are in a standardized metric, computed by dividing the HLM 
gamma coefficient for the outcome variable by the adjusted school-level standard deviation for that 
outcome computed by HLM. 
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Table 6 

Student Engagement in Academic Work: HLM Between-Classroom Model 



Dependent Variable 
Engagement in Academic Work 
Elementary Middle High 


Effects on Mean Between-Classroom Outcome' 








Intercept 


-.65 ** 


.12 


-.74 ** 


Classroom Contextual Features 








Mathematics Classroom 


1.00 ** 


-.70 * 


.92 ** 


Average Level of Authentic Work 


.12 


.46 * 


.66 *** 


Pedagogical Influence 








Progressive Methods 


-.40 * 


-.28 


-.24 


Traditional Methods 


.41 * 


.53 * 


.30 


% Between-Classroom Variance Explained 


64.5 % 


69.2 % 


65.2 % 



'The effects displayed in this table are in a standardized metric, computed by dividing the HLM 
gamma coefficient for each outcome by the adjusted school-level standard deviation for that outcome 
computed by HLM. 
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APPENDIX A 

Social Characteristics of Schools: Restructuring Schools Sample and the Universe of U.S. Public 
Schools Compared' == 



Universe of U.S. Public Schools Restructuring Schools Sample 



Elementary School Level 



Size 447 636 

% Black 16.2 16.4 

% Hispanic 13.5 41.0 

% Free or Reduced Lunch 24.1 51.8 

Middle School Level 



Size 576 646 

% Black 16.1 11-1 

% Hispanic 12.4 10.4 

% Free or Reduced Lunch 19.9 28.7 

High School L<;vel 

Size 714 1050 

% Black 14.6 34.3 

% Hispanic 11-3 13-7 

% Free or Reduced Lunch 11.8 30.0 

All Levels Combined 

Size 522 777 

% Black 16.3 20.6 



% Hispanic 



11.8 21.7 



% Free or Reduced Lunch — 36.8 



Statistics on the Universe of Public Schools were provided by the National Center for tiducation 
Statistics from the Common Core of Data (faxed conununications, March 25, 1994 and May 24, 
1994). The statistics on the percentage of students receiving free and reduced lunch ire based on 
reports from about 65% of all U.S. public schools. Statistics on size and minority enrollment (except 
for all levels combined) are based on reports from 93% of all U.S. public schools. For all levels 
combined, the Office of Civil Rights provided the data on minority enrollment for the universe of 
public schools (S. Tuchman, personal conununication. May 24, 1994). 
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APPENDIX B 



Comparison of Reading and Mathematics Scores by Grade Level for Selected NAEP Items for 
National and Restructuring Study Samples 





i^uniicu vjiduc 


iNationai Average 


Restructuring Schools Average 


Elementary Schools 








Reading 


4 


.66 


.68 


Mathematics 


4 


.64 


.64 


Middle Schools 








Reading 


8 


.59 


.65 


Mathematics 


8 


.44 


.51 


High Schools 








Reading 


12 


.67 


.54 


Mathematics 


12 


.58 


.49 



Construct and Components 



APPENDIX C 
CONSTRUCTION OF VARIABLES 

STUDENTS 



Student Engagement 

« In social studies/mathematics class, how often do you 

try as hard as you can? 
» How often do you complete your assignments for this class? 
« How often do you pay attention in this class? 
0 How often do you feel bored in class? 

Cronbach's a = .70. Composite measure: factor; exponentially 
transformed and standardized. 

Orientation toward School 

(1) Success 

o GPA in English, Social Studies, Mathematics and Science; 
mean of 4 grades 

(2) Alienation 

o I was late for school. 

« I got in trouble for not following school rules. 

Cronbach's a = .33. Composite measure: 
sum of standardized items. 



Secondary students only: 



« I cut or skipped class. 

e I was put on in-school suspension. 

o I was suspended or put on probation from school. 

Cronbach's a (secondary students' items) = .68; 
combined sets of items, a = .73. Composite measure: 
mean of summed scales, standardized; logarithmically 
transformed and standardized. 



Range 


Mean 


SD 


1 -5 


4.4 


.94 


1 - 5 


4.2 


.89 


1 - 5 


4.1 


.84 


1 - D 


Z.J 




5 - 1.8 


0.0 


1.0 


.50-4 


3.0 


.78 


0 - 10 


2.8 


3.3 


n in 


1 Q 




0 - 10 


1.0 




0 - 10 


1.4 


2.8 


0 - 10 


.5 


1.6 


0 - 10 


.4 


1.4 


.18-5.80 


0.0 


1.0 
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Authentic Work 



0 You are asked interesting questions and solve new problems. 1-4 

° You dig deeply into understanding a single topic. 1 - 4 
e You apply the subject to problems and situations 

in life outside of school. 1 - 4 

0 You discuss ideas about the subject with the teacher or students. 1-4 

Cronbach's a = .66. Composite measure: factor. -3.0 - 1.9 

Su pport Structures for Learning 

(1) School 

® In school I often feel "put down" by other students (Rev.). 1 - 4 

o Most of my teachers really listen to what I have to say. 1 - 4 

o I don't feel safe at this school (Rev.). 1 - 4 
o Disruptions by other students get in the way 

of my learning (Rev.). 1 

® My friends and I are treated fairly in this school. 1 



4 
4 



Cronbach's a = .51. Composite measure: mean of standardized components; 
exponentially transformed; standardized. 



(2) Classroom 

e The teacher expects me to do my best all the time, 
o The teacher gives me extra help when 

I don't understand something. 
0 My friends and I help each other with our homework. 

Cronbach's a = .40. Composite measure: factor. 

(3) Parental Support for Learning Index 

Since the beginning of the school year, how often did 
Your parent(s), guardian(s), or other family members: 

o Attend a school meeting 
° Phone or speak to your teacher or counselor 
o Attend a school event in which you participated 
® Act as a volunteer at your school 



1 -4 

1 -4 
1 -4 

-2.67 - 2.04 



0 - 2 
0 - 2 
0 - 2 



2.9 
3.1 

2.7 
3.0 

0.0 



2.7 
3.1 
3.2 

2.4 
3.0 



3.2 

3.4 
2.9 

0.0 



1.0 
.95 
1.1 



.95 
.91 

1.6 
1.0 

1.0 



.93 
.79 
.88 

.89 
.84 



.77 

.79 

.82 

1.0 



.75 
.73 
.80 



Since the begimiing of this school year, how often have you and your parent(s), guardiaii(s), or other 
family members discussed: 



"School activities or events of interest to you 
"Things you have studied in class 



0 - 2 
0 - 2 



1.3 
1.3 



.67 
.66 
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®Your grades 

^Transferring to another school 

Cronbach's o; = .66. Composite measure: sum of 8 components. 

Secondary students only: 

"Selecting courses or programs at school 
'Plans and preparation for ACT or SAT tests 
"Going to college 

Cronbach's a = .59. Composite measure: sum of 3 items. 
Cronbach's a for all parent items = .73. Composite measure: 
(secondary students) mean of standardized scales; 
standardized. 

Personal Background 

(1) Demographic Characteristics 

° Female Gender: Coded Female=l; Male=0 

<» Black: Coded Black=l; Hispanic =0; White, Other =-1 

® Hispanic: Coded Hispanic = l; Black = 0; White, Other=-l 

(2) Socioeconomic Status 
Household Items/Features 

Secondary students only: 
Parents' Education 

(3) Self-Concept 

o I feel good about myself. 

» I am a person of worth, the equal of other people, 
o I am able to do things as well as most other people. 
• At times, I think I am no good at all (rev.), 
o I feel I do not have much to be proud of (rev). 

Cronbach's a = .76. Composite measure: mean, 5 items. 

Secondary students only: 

° On the whole, I am satisfied with myself. 
® I feel useless at times (rev.). 



0-2 
0 - 2 

0 - 16 



0-2 
0-2 
0 - 2 

0 - 6 



-2.8 - 2.5 



Cronbach's a (7 items) 



.86. Composite measure: mean, 7 items. 
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0- 1 
-1 - 1 
-1 - 1 



0 - 10 



10- 18 



1 -4 
1 -4 
1 -4 
1 -4 
1 -4 

1 -4 



1 -4 
1 -4 

1 -4 



1.6 
.48 

7.9 



1.1 
.59 
1.4 

3.1 



0.0 



.51 
.27 

.23 



6.4 



14.2 



3.3 
3.3 
3.3 
2.9 
3.2 

3.2 



3.2 
2.7 

3.2 



.61 
.70 

3.0 



.69 
.71 
.73 

1.6 



1.0 



.50 
.75 
.70 



2.5 



2.3 



.73 
.71 
.70 
.95 
.90 

.57 



.74 
.87 

.57 
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(4) Baseline Test 



® For students in math core classes, the score on items from 
the NAEP math test appropriate to grade level. The score 

is computed as the ratio of items correct to total items. 0-1.0 .48 

® For students in social studies core classes, the average of 
the score on items from the NAEP social studies test 
appropriate to grade level and of the score on a writing 
sample assessed using NAEP rubrics. The score is computed 

as the ratio of items correct to total items. 0-1.0 .57 



.50 



.15 



CLASSROOMS 



Contextual Features 



(1) Average Authentic Work 

Authentic work is aggregated to classroom level. 

(2) Subject Area 

Math: Coded Mathematics = 1 ; Social Studies =0 



2.2 - 3.5 



0-1 



2.9 



.25 



Pedagogical Influence 

(1) Methodis: Traditional 

During a typical week, how much total time (hours) do you spend: 
o lecturing to the class as whole? 
» in drill and practice on basic facts, definitions, 

computations, skills, or procedures? 
« giving tests or quizzes? 

Cronbach's a = .82. Composite measure: factor. 

(2) Methods: Progressive 

During a typical week, how much total time do you spend: 
° leading discussions? 

o providing instruction to individual students? 
« conducting lab periods or other hands-on activities? 
During a typical week, how much total time do students spend: 
o class time for outside activities (field trips, library, 

community experiences, etc.)? 
o working in small groups? 

Cronbach's a = .84. Composite measure: factor. 



0-5 


1.3 


0-5 


1.5 


0-3 


.74 


2.5 - 2.2 


0.0 


0-5 


1.4 


0 - 5 


1.4 


0-5 




0-5 


.55 


0-5 


2.2 



.98 

1.3 
.64 

1.0 



-2.1 -2.1 



0.0 



1.3 
1.3 



.86 
1.5 

1.0 
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Appendix D 



Estimation of parameter variance. The unconditional HLM model, employed as a oneway analysis of 
variance with random effects model, estimates the within- and between-classroom variance in student 
engagement. The combined HLM model for this analysis may be represented as follows: 

Yy = Too + Uoj + ry 

with Yjj the level of engagement (e.g., academic application) for individual i in classroom j; 700 the grand 
mean; Uoj, the classroom effect; and ry, the individual effect. According to this model, the variance of the 
outcome is: 

Var(Yy) = Var(Uoj + ry) = Too + 

with T representing the between-classroom variability and representing the within-classroom variability. 

The within-classroom model. The within-classroom (level one) model for this analysis contains 
individual predictors for academic experiences (authentic work, social support, and demonstrated 
understanding), social integration (attitudes toward school, conduct), and personal background (parental support 
for learning, gender, GPA). To investigate the academic application for student i in classroom j, that is, Yy, 
using the within-classroom model, the equation is: 

Yy = /3oj + ^,jX,y + ^zjXzy + . . . + ^^X^j + ty 

with (3oj the mean for classroom j; jSj^s, the coefficients representing the relationship with academic application 
for each measured student characteristic in classroom j; and ry the within-classroom error, assumed to be 
normally distributed (0,a^). In this model, when the relationship of the measured characteristics of students to 
academic application did not vary significantly between classrooms, the parameters (except for dummy 
variables) are centered around the grand mean (Xy - X..), and they are fixed, i.e., set to 0, rather than allowed 
to vary randomly between classrooms. Because the relationship between authentic work and academic 
application varies significantly between classrooms, it is centered around the classroom mean (Xy -X.j) and 
modeled as a randomly varying slope. 

The between-classroom model. The between classroom (level two) equation models academic 
application, adjusted for student characteristics, as a function of teachers' professional practice (disciplinary 
competence and methods), an explanatory contextual feature (the average level of authentic work), and controls 
(level, subject area, and average prior achievement: 

i3jt = 7ok + TikW.j -t- 7^W2j + ... + 7p,Wpj + Uj,. 

with Wj representing a between-classroom predictor. 
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